Evidence Aggregation in Hierarchical Evidential Reasoning

Several reasoning tasks need to scale over the volume of evidence and the entities of interest. A technique used by statisticians is to represent a collection of evidence by the sufficient statistics of the data. This reduces the computation burden on the BN model significantly by representing a large set of records (and therefore nodes) with a single node. However, this procedure is not general in terms of data types and model parameters. Similar to the notion of the sufficient statistics, many real world applications of Bayesian Networks use this common pattern of use where a set of evidence is approximated by an aggregate score of the collective. In some applications the approximation is justified by appealing to the sufficient statistics argument. However, in others it is presented only as a heuristic for computational tractability. In this paper we review the pattern of use where a collection is approximated by an aggregate score and the sufficient statistics argument in the context of Bayesian reasoning. Further, we present use of the same technique where the sufficient statistics notion is not applicable. It seems like there needs to be a theoretical justification of the approximation based on a generalization of sufficient statistics. We present two applications where we have used this pattern of use and with remarkable effectiveness. The applications are in (1) cyber security, and (2) resource provisioning in distributed systems.